Papers with sequence-to-sequence task
AutoTrain: No-code training for state-of-the-art models (2024.emnlp-demo)
Copied to clipboard
| Challenge: | AutoTrain is an open-source, no code tool/library which can be used to train models on custom datasets. |
| Approach: | They propose an open-source, no-code tool/library to train models on custom datasets. |
| Outcome: | The open-source, no-code tool/library can be used to train models on custom datasets. |
Systematic Generalization in Language Models Scales with Information Entropy (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks for assessing compositional behavior are unclear on how to measure the difficulty of a systematic generalization problem. |
| Approach: | They propose a framework for measuring entropy in a sequence-to-sequence task and a method for measuring it. |
| Outcome: | The proposed framework scales with the entropy of the distribution of component parts in the training data. |
BiTIIMT: A Bilingual Text-infilling Method for Interactive Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing IMT systems relying on lexical constrained decoding (LCD) are limited in translation efficiency and quality due to LCD. |
| Approach: | They propose a novel interactive neural machine translation system that uses lexical constraints to decode missing words in a manually revised translation. |
| Outcome: | The proposed system performs significantly better and faster than state-of-the-art IMT on three translation tasks. |
Sequence-to-Sequence Knowledge Graph Completion and Question Answering (2022.acl-long)
Copied to clipboard
| Challenge: | Knowledge graph embedding (KGE) models represent each entity and relation of a knowledge graph (KG) with low-dimensional embeddable vectors. |
| Approach: | They propose to use an off-the-shelf encoder-decoder Transformer model to generate a knowledge graph embedding model that can be used for KG link prediction and incomplete KG question answering. |
| Outcome: | The proposed model outperforms baselines on multiple large-scale datasets without extensive hyperparameter tuning. |
Graph-Based Decoding for Task Oriented Semantic Parsing (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing paradigms for semantic parsing are sequence-to-sequence and AMR parsers. |
| Approach: | They propose to formulate parsing as a sequence-to-sequence task using graph-based decoding techniques developed for syntactic parsers. |
| Outcome: | The proposed approach is competitive with sequence decoders on the standard setting and offers significant improvements in data efficiency and data availability. |
Byte-Level Grammatical Error Correction Using Synthetic and Curated Corpora (2023.acl-long)
Copied to clipboard
Svanhvít Lilja Ingólfsdóttir, Petur Ragnarsson, Haukur Jónsson, Haukur Simonarson, Vilhjalmur Thorsteinsson, Vésteinn Snæbjarnarson
| Challenge: | Spelling mistakes due to typos and rushed writing, nonstandard punctuation and spelling, and grammatical and stylistic issues are common to almost everyone who writes any kind of text. |
| Approach: | They propose to use a common subword unit vocabulary and byte-level encoding to fine tune two subword-level models and one byte level model on hand-corrected error corpora. |
| Outcome: | The proposed model improves accuracy for spelling and grammatical errors and more complex errors. |
Composing Ci with Reinforced Non-autoregressive Text Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to compose Ci are limited in handling the constraints of tune patterns . authors propose a non-autoregressive approach to generate Ci using a synchronous process . |
| Approach: | They propose to compose Ci using a non-autoregressive approach that takes into account rigid formats . they propose to apply reinforcement learning to the generation process with rigid constraints . |
| Outcome: | The proposed method outperforms baselines and previous studies on a Ci dataset . it allows the model to perform synchronous generation while maintaining the format and content requirement. |
Video-Grounded Dialogues with Pretrained Generation Language Models (2020.acl-main)
Copied to clipboard
| Challenge: | Pre-trained language models have shown success in improving downstream NLP tasks . pre-tuned models capture textual dependencies in text data of rich semantics . |
| Approach: | They propose a framework for improving video-grounded dialogue by extending GPT-2 models . they propose to combine visual and textual representation into a structured sequence . |
| Outcome: | The proposed framework improves audio-visual scene-aware dialogues benchmark on AVSD . it is based on a large pre-trained GPT-2 network and can generate natural responses . |
Exploring the Synergy of Dual-path Encoder and Alignment Module for Better Graph-to-Text Generation (2024.lrec-main)
Copied to clipboard
| Challenge: | KG-to-text generation model lacks explicit graph-text alignment strategy due to discrepancy between textual and structure information. |
| Approach: | They propose a synergetic knowledge graph-to-text model with a dual-path encoder, alignment module and guidance module to solve these problems. |
| Outcome: | The proposed model achieves competitive performance on three benchmark datasets. |
Revisiting Relation Extraction in the era of Large Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Standard supervised approaches to RE learn to tag tokens comprising entity spans and then predict the relationship between them. |
| Approach: | They propose to use large language models for RE to evaluate their performance . they use GPT-3 and Flan-T5 large to train RE . |
| Outcome: | The proposed model outperforms existing models on a sequence-to-sequence task under varying levels of supervision. |
SLICEFORMER: Static Program Slicing Using Language Models With Dataflow-Aware Pretraining and Constrained Decoding (2026.acl-long)
Copied to clipboard
| Challenge: | Static program slicing is a software engineering technique for isolating code relevant to specific variables. |
| Approach: | They propose a new approach that reformulates static program slicing as a sequence-to-sequence task using small language models such as CodeT5+. |
| Outcome: | The proposed approach improves on Java and Python program slicing benchmarks with up to 22% gain in ExactMatch. |